Skip to content

On folder move execute only one UPDATE query for all nested items.#6211

Closed
allexzander wants to merge 16 commits intomasterfrom
feature/single-sql-query-on-move
Closed

On folder move execute only one UPDATE query for all nested items.#6211
allexzander wants to merge 16 commits intomasterfrom
feature/single-sql-query-on-move

Conversation

@allexzander
Copy link
Contributor

@allexzander allexzander commented Nov 9, 2023

So, the idea here is to not execute INSERT query for every nested item of moved folder (there can be thousands of items and this is very very slow and also brings an issue of user moving/renaming the same folder multiple times, causing local database records to become messed up and virtual files being removed on a later sync). Instead, the UPDATE query is executed, which is very fast. Tested on 1000 files so far so good.
Let's keep it draft for now until I have time to properly finalize it.

Should fix #4636 or at least make it much less likely to happen.

@allexzander allexzander force-pushed the feature/single-sql-query-on-move branch 2 times, most recently from 9b38399 to 982df39 Compare November 13, 2023 16:29
@allexzander allexzander reopened this Nov 15, 2023
@allexzander allexzander force-pushed the feature/single-sql-query-on-move branch from 42276df to 08e1958 Compare November 16, 2023 00:31
@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 16, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot E 2 Security Hotspots
Code Smell B 40 Code Smells

83.5% 83.5% Coverage
0.0% 0.0% Duplication

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

@allexzander allexzander force-pushed the feature/single-sql-query-on-move branch from ccc72ba to cb9e5ac Compare January 10, 2024 15:58
@allexzander allexzander force-pushed the feature/single-sql-query-on-move branch from 147c170 to 35e5446 Compare January 29, 2024 20:05
@allexzander allexzander marked this pull request as ready for review January 29, 2024 20:12
@allexzander allexzander force-pushed the feature/single-sql-query-on-move branch from 35e5446 to 18008eb Compare February 14, 2024 20:54
@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 14, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
2 Security Hotspots
64 New Code Smells (required ≤ 0)
C Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

idea Catch issues before they fail your Quality Gate with our IDE extension SonarLint SonarLint

@allexzander allexzander mentioned this pull request Feb 18, 2024
Closed
@allexzander allexzander force-pushed the feature/single-sql-query-on-move branch from 18008eb to 122ae9c Compare April 24, 2024 18:27
@sonarqubecloud
Copy link

sonarqubecloud bot commented Apr 24, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
2 Security Hotspots
C Maintainability Rating on New Code (required ≥ A)
74 New Code Smells (required ≤ 0)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

@mgallien mgallien force-pushed the feature/single-sql-query-on-move branch from 122ae9c to 2a7adfe Compare August 13, 2024 07:44
@nextcloud-desktop-bot
Copy link

nextcloud-desktop-bot commented Aug 13, 2024

AppImage file: nextcloud-PR-6211-2a7adfe766f6258e420c2b51dee79447684c40c2-x86_64.AppImage

To test this change/fix you can simply download above AppImage file and test it.

Please make sure to quit your existing Nextcloud app and backup your data.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Aug 13, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
2 Security Hotspots
C Maintainability Rating on New Code (required ≥ A)
81 New Code Smells (required ≤ 0)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

@Rello Rello added this to the 3.16.0 milestone Dec 20, 2024
mgallien
mgallien requested changes Feb 10, 2025
View reviewed changes
Copy link
Collaborator

@mgallien mgallien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conflicts need to be fixed
most probably tests should be done (maybe automated and manual) to ensure no regressions

@camilasan camilasan modified the milestones: 3.16.0, 3.17.0 Feb 10, 2025
@mgallien mgallien force-pushed the feature/single-sql-query-on-move branch from 2a7adfe to 95a59f9 Compare March 14, 2025 16:16
@mgallien mgallien requested a review from nilsding as a code owner March 14, 2025 16:16
@mgallien mgallien force-pushed the feature/single-sql-query-on-move branch 2 times, most recently from e952db6 to 12b9724 Compare March 14, 2025 16:54
@mgallien mgallien force-pushed the feature/single-sql-query-on-move branch from 12b9724 to 46f2a52 Compare March 14, 2025 16:55
@mgallien mgallien force-pushed the feature/single-sql-query-on-move branch from 46f2a52 to 02e600c Compare March 31, 2025 14:36
allexzander added 15 commits April 9, 2025 16:38
@allexzander @mgallien
On folder move execute only one UPDATE query for all nested items.
92c08e9 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Iteration. Fixing tests.
b72c44f 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Iteration. Improved move optimization logic. Also account for rename …
debf697 
…target such that it is the same move.

Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
iteration.
eb9b4e9 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Iteration.
f5a1251 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Fixed incorrect move logic and tests.
042ad18 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Remove unnecessary changes.
d617ee6 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Refactoring.
92be906 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Do not update pinstates and inode on move for WindowsCfApi mode.
5a89042 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Refactoring.
dae53bf 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Fix compile issues.
03b939b 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Added some todos.
de10f43 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Fix a typo.
d2c57e4 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Unit tests. SyncJournal db. Test move with a single query.
35ca44d 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@allexzander @mgallien
Give a proper name to a parent path update method in SyncJournal.
5589089 
Signed-off-by: alex-z <blackslayer4@gmail.com>
@mgallien mgallien force-pushed the feature/single-sql-query-on-move branch 2 times, most recently from c79ec75 to 38ad19e Compare April 9, 2025 17:03
@mgallien
some clean up
3551c3e 
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
@mgallien mgallien force-pushed the feature/single-sql-query-on-move branch from 38ad19e to 3551c3e Compare April 9, 2025 17:11
@github-actions
Copy link

github-actions bot commented Apr 9, 2025

Artifact containing the AppImage: nextcloud-appimage-pr-6211.zip

SHA256 checksum: 92bc40bc951115f561d36a9ca389ee059899432432ce54aae601d080b4fdd5ac

To test this change/fix you can download the above artifact file, unzip it, and run it.

Please make sure to quit your existing Nextcloud app and backup your data.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Apr 9, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
2 Security Hotspots
89 New Code Smells (required ≤ 0)
C Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@mgallien mgallien modified the milestones: 3.17.0, 3.18.0 May 26, 2025
@mgallien mgallien closed this Jul 15, 2025
@Rello Rello deleted the feature/single-sql-query-on-move branch February 3, 2026 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mgallien mgallien mgallien requested changes

@claucambra claucambra claucambra approved these changes

@camilasan camilasan Awaiting requested review from camilasan camilasan is a code owner

@nilsding nilsding Awaiting requested review from nilsding nilsding is a code owner

Assignees

@mgallien mgallien

Labels

2. developing

Projects

None yet

Milestone

4.0.0

Development

Successfully merging this pull request may close these issues.

[Bug]: Some virtual files cannot be sync'd: "Conflict when uploading a file. It's going to get removed!"

6 participants

@allexzander @nextcloud-desktop-bot @mgallien @claucambra @camilasan @Rello